We consider the problem of variational Bayesian inference in a latent variable model where a (possibly complex) observed stochastic process is governed by the solution of a latent stochastic differential equation (SDE).

We would like to thank all the reviewers for their effort, and their thoughtful comments. Being formal, it should be "the gradient associated to the pullback of We will change it to "on which standard convergence results still apply". Thm 4.3 We will change "is equivalent" to The same can be said about higher order methods. We chose not to mention them in the main paper for simplicity. In l.138 we do mean "in almost all the manifold" in a measure-theoretical sense with respect to a measure induced by These two things indeed deserve a clarifying footnote.

Through the use of convolution layers, Convolutional Neural Networks (CNNs) have a built-in understanding of locality and translational symmetry that is inherent in many learning problems.

Meanwhile, we make the kernel locally supported without breaking the equivariance.

We consider the problem of variational Bayesian inference in a latent variable model where a (possibly complex) observed stochastic process is governed by the solution of a latent stochastic differential equation (SDE).

Meanwhile, we make the kernel locally supported without breaking the equivariance.

We present a general theory of Group equivariant Convolutional Neural Networks (G-CNNs) on homogeneous spaces such as Euclidean space and the sphere. Feature maps in these networks represent fields on a homogeneous base space, and layers are equivariant maps between spaces of fields. The theory enables a systematic classification of all existing G-CNNs in terms of their symmetry group, base space, and field type. We also answer a fundamental question: what is the most general kind of equivariant linear map between feature spaces (fields) of given types? We show that such maps correspond one-to-one with generalized convolutions with an equivariant kernel, and characterize the space of such kernels.

This paper presents a novel framework for non-linear equivariant neural network layers on homogeneous spaces. The seminal work of Cohen et al. on equivariant $G$-CNNs on homogeneous spaces characterized the representation theory of such layers in the linear setting, finding that they are given by convolutions with kernels satisfying so-called steerability constraints. Motivated by the empirical success of non-linear layers, such as self-attention or input dependent kernels, we set out to generalize these insights to the non-linear setting. We derive generalized steerability constraints that any such layer needs to satisfy and prove the universality of our construction. The insights gained into the symmetry-constrained functional dependence of equivariant operators on feature maps and group elements informs the design of future equivariant neural network layers. We demonstrate how several common equivariant network architectures - $G$-CNNs, implicit steerable kernel networks, conventional and relative position embedded attention based transformers, and LieTransformers - may be derived from our framework.

I would give an accept score if I were able to have a look at the new version and be happy with it (as is possible in openreview settings for example). However since improving the presentation usually takes a lot of work and it is not possible for me to verify in which way the improvements have actually been implemented, I will bump it to a 5. I do think readability and clarity is key for impact as written in my review, which is the main reason I gave a much lower score than other reviewers, some of whom have worked on exactly this intersection of algebra and G-CNNs themselves and provided valuable feedback on the content from an expert's perspective. The following comments are based on the reviewer's personal definition of clarity and good quality of presentation: that most of the times when following the paper from start to end it is clear to the reader why each paragraph is written and how it links to the objective of the main results of the paper, here claimed e.g. in the last sentence to be the development of new equivariant network architectures. The paper is one long lead-up of three pages of definitions of mathematical terms and symbols to the theorems in section 6 on equivariant kernels which represent the core results of the paper. In general, I appreciate rigorous frameworks which generalize existing methods, especially if they provide insight and enable the design of an arbitrary new instance that fits in the framework (in this case transformations on arbitrary fields).